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application: 



Listing of Claims: 



1 (amended). A method of associating additional information with a video including a 
plurality of frames comprising: 

(a) identifying at least one of said frames representative of a picture 
composed of a plurality of pixels : 

(b) providing a descriptive stream separate from said video; 

(c) including said additional information in said descriptive stream related to 
said at least one of said frames; 

(d) providing said video for displaying on a display; and 

(e) selectively providing said additional information to a viewer approximately 



2(original). The method of claim 1 wherein said additional information includes at 
least one of an object index, a textual description, a voice annotation, an image feature, an 
object link, a URL link, and a Java applet. 

3(original). The method of claim 1 wherein said identifying is an object within said 

frame. 

4(original). The method of claim 1 where said descriptive stream is related to a 
plurality of said frames. 

5(original). The method of claim 4 wherein said at least one of said frames are in 
sequential order in said video. 



the time of said providing said video where said additional information is 
an object depicted by said picture by said pixels . 
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6(original). The method of claim 4 wherein said at least one of said frames are in 
nonsequential order in said video. 

7(original). A method of claim 3 wherein said additional information is related to said 

object. 

8(original). The method of claim 1 wherein said descriptive stream includes an index 
synchronizing said video with said descriptive stream. 

9(original). The method of claim 1 wherein said descriptive stream includes copyright 
information. 

10(original). The method of claim 1 wherein said descriptive stream is encoded 
separately from said video. 

1 1 (original). The method of claim 10 wherein said video is decoded in the same 
manner independently of whether said descriptive stream is provided. 

12(original). The method of claim 1 1 wherein said video is at least one of MPEG-2 
and television broadcast format. 

13(original). The method of claim 1 wherein said additional information is presented to 
said viewer on a remote control. 

14(original). The method of claim 1 wherein an audible signal indicates the availability 
of said additional information. 

15(original). The method of claim 1 wherein a visual signal indicates the availability of 
said additional information. 
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16(original). The method of claim 7 wherein said additional information includes 
textual based information related to said object. 

17(original). The method of claim 7 wherein said additional information includes 
audible information related to said object. 

18(original). The method of claim 7 wherein said additional information includes image 
features comprising at least one of texture, shape, dominant color, and a motion model related 
to said object. 

19(original). The method of claim 7 wherein said additional information includes links 
to at least one of other objects and frames within said video. 

20(original). The method of claim 7 wherein said additional information includes 
program instructions related to said object. 

21(amended).A video system comprising: 

(a) an encoder that includes additional information within a video stream 
including a video including a plurality of frames representative of a picture 
composed of a plurality of pixels , where said additional information is 
related to at least one of said frames; 

(b) a receiver that receives said video and said additional information, and 
« 

said receiver decodes said video in the same manner independently of 
whether said additional information is provided; 

(c) a display for displaying said video; and 

(d) a trigger mechanism for selectively presenting said additional information 
to a viewer at approximately the time of presenting said frames to said 
viewer where said additional information is an object depicted by said 
picture by said pixels . 
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22(original). The system of claim 21 , further comprising: 

(a) a transmitter for transmitting said video signal and said additional 
information; and 

(b) a receiver for receiving said video signal and said additional information. 

23(original). The system of claim 22 wherein said encoder is at least one of a video 
camera and a computer. 

24(original). The system of claim 21 wherein said trigger mechanism is located in a 
remote control device. 

25(original). The system of claim 21 wherein said additional information is provided by 
a remote control device. 

26(original). The method of claim 21 wherein said additional information is related to 
an object within said frame and includes links to at least one of other objects and frames within 
said video. 

27(original). The method of claim 21 wherein said additional information is related to 
an object within said frame and includes program instructions related to said object. 

28(original). The method of claim 21 wherein said additional information is related to 
an object within said frame and includes textual based information related to said object. 

29(original). The method of claim 21 wherein said additional information is related to 
an object within said frame and includes audible information related to said object. 
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30(original). The method of claim 21 wherein said additional information is related to 
an object within said frame and includes image features comprising at least one of texture, 
shape, dominant color, and a motion model related to said object. 



31 (amended once). A system for presenting information comprising: 

(a) a unitary file containing both an image representative of a picture 
composed of a plurality of pixels and additional information associated 
with said image; 

(b) a selection mechanism that permits the selection of graphical objects 
depicted by said picture by said pixels in said image for which said 
additional information is related thereto; and 

(c) a presentation mechanism that provides said additional information to a 
viewer in response to selecting said object depicted by said picture by 
said pixels . 



32(original). The system of claim 31 wherein said file includes said image followed by 
said additional information. 

33(original). The system of claim 32 wherein said image and said additional 
information are separated by a marker indicating the end of said image. 

34(original). The system of claim 33 wherein an image viewer which does not 
recognize said additional information will display said image properly and recognize said marker 
as indicating the end of said image. 

35(original). The system of claim 34 wherein said image is in a JPEG format. 



36(original). The system of claim 31 wherein said additional information is organized 
in at least two layers comprising: 
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(a) a first layer containing information describing the location of objects within 
said image; and 

(b) a second layer containing additional information regarding said objects 
within said image, where said first layer contains fewer bytes than said 
second layer. 



37(original). The system of claim 36 wherein said second layer follows said first layer, 
which in turn follows said image file. 

38(original). The system of claim 36 wherein said first layer contains a length identifier 
describing the length of said first layer. 

39(original). The system of claim 36 wherein said first layer contains a number of 
objects identifier describing the number of objects identified by said first layer. 

40(original). The system of claim 36 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 

41 (original). The system of claim 36 wherein said first layer contains a first definition 
of the outline of an object of said image. 

42(original). The system of claim 36 wherein said second layer contains a length 
identifier describing the length of said second layer. 

43(original). The system of claim 36 wherein said second layer contains an array of 
offsets that identify the start of each data item. 
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44(original). The system of claim 41 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition more closely 
approximates the outline of said object than said first definition. 

45(original). The system of claim 41 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition contains more 
bytes than said first definition. 

46(original). The system of claim 36 wherein said second layer includes sound data 
related to said object. 

47(original). The system of claim 36 wherein said second layer includes HTML meta 
tags related to said object. 

48(original). The system of claim 36 wherein said second layer includes textual 
annotations related to said object. 

49(original). The system of claim 36 wherein said second layer includes an HTML 
page to be rendered. 

50(original). The system of claim 36 wherein said second layer includes a Java Applet 
related to said object. 

51 (original). The system of claim 36 wherein said second layer includes a color 
histogram. 

52(original). The system of claim 36 wherein said second layer includes data related 
to the conditions under which said image was created including at least one of lighting, camera 
settings, and time of acquisition. 
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53(original). The system of claim 36 wherein said second layer includes data related 
to information for reproducing said image including at least one of cropping information, paper 
type, camera settings, and image production settings. 

54(original). The system of claim 36 wherein said second layer includes another 
image to be superimposed upon said image. 

55(original). The system of claim 36 wherein said second layer includes data 
regarding the author of said image. 

56(original). The system of claim 36 wherein said second layer includes copyright data 
regarding the copyright of said image. 

57(original). The system of claim 56 said copyright data is encoded. 

58(original). The system of claim 36 wherein said second layer includes information 
regarding how said image should be viewed. 

59(amended once). A method of presenting information comprising: 

(a) displaying an image from a unitary file containing both an image 
representative of a picture composed of a plurality of pixels and additional 
information associated with said image; 

(b) selecting an object depicted by said picture by said pixels in said image 
for which said additional information is related thereto; and 

(c) providing said additional information to a viewer in response to selecting 
said object depicted by said picture by said pixels . 

60(original). The method of claim 59 wherein said file includes said image followed by 
said additional information. 
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61 (original). The method of claim 60 wherein said image and said additional 
information are separated by a marker indicating the end of said image. 

62(original). The method of claim 61 wherein an image viewer which does not 
recognize said additional information will display said image properly and recognize said marker 
as indicating the end of said image. 

63(original). The method of claim 62 wherein said image is in a JPEG format. 

64(original). The method of claim 59 wherein said additional information is organized 
in at least two layers comprising a first layer containing information describing the location of 
objects within said image and a second layer containing additional information regarding said 
objects within said image, where said first layer contains fewer bytes than said second layer. 

65(original). The method of claim 64 wherein said second layer follows said first layer, 
which in turn follows said image file. 

66(original). The method of claim 64 wherein said first layer contains a length identifier 
describing the length of said first layer. 

67(original). The method of claim 64 wherein said first layer contains a number of 
objects identifier describing the number of objects identified by said first layer. 

68(original). The method of claim 64 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 

69(original). The method of claim 64 wherein said first layer contains a first definition 
of the outline of an object of said image. 
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70(original). The method of claim 64 wherein said second layer contains a length 
identifier describing the length of said second layer. 

71 (original). The method of claim 64 wherein said second layer contains an array of 
offsets that identify the start of each data item. 

72(original). The method of claim 69 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition more closely 
approximates the outline of said object than said first definition. 

73(original). The method of claim 69 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition contains more 
bytes than said first definition. 

74(original). The method of claim 64 wherein said second layer includes sound data 
related to said object. 

75(original). The method of claim 64 wherein said second layer includes HTML meta 
tags related to said object. 

76(original). The method of claim 64 wherein said second layer includes textual 
annotations related to said object. 

77(original). The method of claim 64 wherein said second layer includes an HTML 
page to be rendered. 

78(original). The method of claim 64 wherein said second layer includes a Java Applet 
related to said object. 
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79(original). The method of claim 64 wherein said second layer includes a color 
histogram. 

80(original). The method of claim 64 wherein said second layer includes data related 
to the conditions under which said image was created including at least one of lighting, camera 
settings, and time of acquisition. 

81 (original). The method of claim 64 wherein said second layer includes data related 
to information for reproducing said image including at least one of cropping information, paper 
type, camera settings, and image production settings. 

82(original). The method of claim 64 wherein said second layer includes another 
image to be superimposed upon said image. 

83(original). The method of claim 64 wherein said second layer includes data 
regarding the author of said image. 

84(original). The method of claim 64 wherein said second layer includes copyright 
data regarding the copyright of said image. 

85(original). The method of claim 64 said copyright data is encoded. 

86(original). The method of claim 64 wherein said second layer includes information 
regarding how said image should be viewed. 

87(amended once). A method for creating an image file comprising: 

(a) providing an image representative of a picture composed of a plurality of 
Pixels; ^ a 

(b) selecting an object depicted by said picture by said pixels in said image; 
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(c) 
(d) 



providing additional information related to said object; and 

storing both said image and said additional information in a unitary file. 



88(original). The method of claim 87 wherein said file includes said image followed by 
said additional information. 

89(original). The method of claim 88 wherein said image and said additional 
information are separated by a marker indicating the end of said image. 

90(original). The method of claim 89 wherein an image viewer which does not 
recognize said additional information will display said image properly and recognize said marker 
as indicating the end of said image. 

91 (original). The method of claim 90 wherein said image is in a JPEG format. 

92(original). The method of claim 87 wherein said additional information is organized 
in at least two layers comprising a first layer containing information describing the location of 
objects within said image and a second layer containing additional information regarding said 
objects within said image, where said first layer contains fewer bytes than said second layer. 

93(original). The method of claim 92 wherein said second layer follows said first layer, 
which in turn follows said image file. 

94(original). The method of claim 92 wherein said first layer contains a length identifier 
describing the length of said first layer. 

95(original). The method of claim 92 wherein said first layer contains a number of 
objects identifier describing the number of objects identified by said first layer. 
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96(original). The method of claim 92 wherein said first layer contains a number of data 
identifier describing the number of data items associated with a particular said object. 

97(original). The method of claim 92 wherein said first layer contains a first definition 
of the outline of an object of said image. 

98(original). The method of claim 92 wherein said second layer contains a length 
identifier describing the length of said second layer. 

99(original). The method of claim 92 wherein said second layer contains an array of 
offsets that identify the start of each data item. 

100(original). The method of claim 97 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition more closely 
approximates the outline of said object than said first definition. 

101 (original). The method of claim 97 wherein said second layer contains a second 
definition of the outline of said object of said image, where said second definition contains more 
bytes than said first definition. 

102(original). The method of claim 92 wherein said second layer includes sound data 
related to said object. 

103(original). The method of claim 92 wherein said second layer includes HTML meta 
tags related to said object. 

104(original). The method of claim 92 wherein said second layer includes textual 
annotations related to said object. 
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105(original). The method of claim 92 wherein said second layer includes an HTML 
page to be rendered. 

106(original). The method of claim 92 wherein said second layer includes a Java Applet 
related to said object. 

107(original). The method of claim 92 wherein said second layer includes a color 
histogram. 

108(original). The method of claim 92 wherein said second layer includes data related 
to the conditions under which said image was created including at least one of lighting, camera 
settings, and time of acquisition. 

109(original). The method of claim 92 wherein said second layer includes data related 
to information for reproducing said image including at least one of cropping information, paper 
type, camera settings, and image production settings. 

1 10(original). The method of claim 92 wherein said second layer includes another 
image to be superimposed upon said image. 

1 1 1 (original). The method of claim 92 wherein said second layer includes data 
regarding the author of said image. 

1 12(original). The method of claim 92 wherein said second layer includes copyright 
data regarding the copyright of said image. 

1 1 3(original). The method of claim 92 said copyright data is encoded. 
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1 14(original). The method of claim 92 wherein said second layer includes information 
regarding how said image should be viewed. 

1 15(original). The method of claim 92 wherein said first layer is transmitted from a first 
computer to a second computer together with said image. 

1 16(original). The method of claim 115 wherein portions of said second layer are 
transmitted from said first computer to said second computer upon request by said first 
computer. 

1 17(original). The method of claim 1 16 wherein said request is in response to a user 
selecting an object within said image. 
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